Distributed Stream Processing with DUP
نویسندگان
چکیده
This paper introduces the DUP System, a simple framework for parallel stream processing. The DUP System enables developers to compose applications from stages written in almost any programming language and to run distributed streaming applications across all POSIXcompatible platforms. Parallel applications written with the DUP System do not suffer from many of the problems that exist in traditional parallel languages. The DUP System includes a range of simple stages that serve as general-purpose building blocks for larger applications. This work describes the DUP assembly language, the DUP architecture and some of the stages included in the DUP run-time library. We then present our experiences with parallelizing and distributing the ARB Project, a package of tools for RNA/DNA sequence database handling and analysis.
منابع مشابه
Relation between Working Memory Capacity and Auditory Stream Segregation in Children with Auditory Processing Disorder
Background: This study assessed the relationship between working memory capacity and auditory stream segregation by using the concurrent minimum audible angle in children with a diagnosed auditory processing disorder (APD).Methods: The participants in this cross-sectional, comparative study were 20 typically developing children and 15 children with a diagnosed APD (age, 9–11 years) according to...
متن کاملSynergy: Sharing-Aware Component Composition for Distributed Stream Processing Systems
Many emerging on-line data analysis applications require applying continuous query operations such as correlation, aggregation, and filtering to data streams in real-time. Distributed stream processing systems allow in-network stream processing to achieve better scalability and quality-of-service (QoS) provision. In this paper we present Synergy, a distributed stream processing middleware that ...
متن کاملA Quality-Centric Data Model for Distributed Stream Management Systems
It is challenging for large-scale stream management systems to return always perfect results when processing data streams originating from distributed sources. Data sources and intermediate processing nodes may fail during the lifetime of a stream query. In addition, individual nodes may become overloaded due to processing demands. In practice, users have to accept incomplete or inaccurate quer...
متن کاملDistributed Reactive Stream Processing
Reactive programming paradigm successfully overcomes the limitations of observer pattern which has traditionally been used for developing event-driven distributed systems. Due to its declarative style, compositionality and automatic management of dependencies, reactive programming offers a promising new way for building complex distributed data-flow systems. This article outlines some open chal...
متن کاملQuery Optimization for Distributed Data Streams
With the recent explosive growth of sensors and instruments, data-driven stream applications are emerging as a new field. Query optimization for such high performance stream applications has not been extensively studied, especially its core component, the cost model. We observe that the cost model for stream query processing should consider two aspects: output rate and computation cost. However...
متن کامل